This invention has utility with the remote control of a wide variety of tele-operated robots and vehicles. While the description provided herein describes the methods and systems of the present invention in relationship to a specific mobile robot, the invention is not so limited. One of skill in the art will recognize that the methods and systems described herein have broad applicability for the remote control of robotic devices.
As robots become increasingly common and capable, there will be an increasing need for an intuitive method of and system for remotely controlling the robots. For example, users may have remote access to robots with which they are otherwise unfamiliar. Just as a licensed driver feels comfortable operating a rental car she has never before encountered, so should she feel comfortable remotely operating an unfamiliar robot; to date, this has not been possible.
While in many situations a robot can be operated locally with the user in visual contact with the robot, in many other situations it is advantageous to have the robot tele-operated. For example, in situations where the robot must operate in hazardous or dangerous conditions—e.g., the transport of hazardous materials, search and rescue operations, military and law enforcement applications—tele-operation of the robot is particularly beneficial.
In some existing systems, a camera is carried by a robot and pictures of the view seen by the camera are transmitted by a communications link to a remote control station and reproduced there on a display screen to give the operator some visual information on the vehicle's environment. In yet other existing systems, users painstakingly build maps or detailed floor plans of the robot's environment in order to remotely navigate. Because of compounding errors generated by such systems, these systems are often inadequate.
The most difficult systems to use are interfaces in which the user specifies a velocity, effectively using a joystick-like interface. This approach suffers over communications lines, since there is a time lag between when a picture is taken and when a user can see it, and again there is a time lag between when a user stops a movement with a joystick and when that command is received by the robot. Typically this kind of interface suffers from “overshooting,” where the user stops commanding a motion when they see the image that shows the robot at its desired location. However, since that image has aged, the robot has already actually overshot the desired location. Since the command to stop moving also takes time to arrive at the destination, the robot continues to overshoot while this command is in transit.
One solution to the overshooting problem is the inclusion of simple, clickable arrows on an interface to command a fixed amount of movement by specifying travel time or distance. This simple interface has the desirable characteristic that it provides an absolute motion command to the robot which will not suffer from time lag issues; however, this interface provides limited functionality.
Yet another possible solution includes using fixed cameras that point to an immovable target and then allowing a user to select locations for a robot to move to within the fixed image. This solution lacks the ability to arbitrarily position and rotate the camera in three-dimensional space. In addition, this solution requires placing cameras in all locations to which the robot can travel, and therefore is an inflexible and expensive solution.
Because existing systems are often difficult to control, additional solutions have been proposed. For example, in U.S. Pat. No. 6,108,031, a user is given “virtual reality” glasses (or a headset) to allow three-dimensional information to be transmitted to the user. Using this enhanced visual information, the user then remotely manipulates the vehicle using a control box.
There are, however, limitations to these methods of remotely controlling a robot or vehicle. As mentioned above, in many of these cases, it is assumed that real-time visual information is being transmitted from the camera to the user and that the user is able to transmit real-time control information back to the robot. For certain types of communication links, however, such real-time communication is not possible. Specifically, Internet connections can vary dramatically by the speed of the connection (e.g. DSL, cable modem, dial-up connections) and by the current level of Internet traffic. Therefore, for Internet-connected robots, such real-time transmission cannot be guaranteed.